DOCUMSNT RBSUME 



CS 20t3 1B3 

Worrell, Dcnna K. 

D^finin^ tho Basic Writing Student ny Count. 
Mar 8 1 

lOp.; Paper presfc^nted at ^hu Annu^i. i-iti^^^uj oi ztv; 
Confurer^ce on College Composition ina co:uauin ic^ti on 
(32nd, Dallas, Tx, rtarch 26-23, l^dl). 

MF01/PC01 Plus Postage, 

college Englijh* ^Coilegi- freshtnun^ ^i^rror nnai/zsi? 
(Languaga) ; Evaluation Method hijnar Ea^OaZLOM 
*Hoiisi;ic Evaj^uation; ^Mini;n.. . com y^t^ncy le s^rng ; 
a^fcedial Instruction; ^Student Plac^uit^nt; i% r Iting 
Eviiluation: ^Hriting Instruction; MZkzLViq SkiXIs 



In order ^o establish compatency yaseid v^nt^iance 
reyuireiaents for rrBShman com posii:ion , a niidweatHrn un^vy^sity 
sampled six freshmau composition and four basic writing cxaases tor 
errors in thair essay writing, ACT English and social s^uaies scores 
and several reading scores were also gathered. All this uata waS 
computer analyzed by the multiple regression procadure, i/ith the 
final holistic rating as the criterion \^ariable- aesuxts inaicated 
thm aolistic rating as the best indicator of succass m tn& creshLnan 
writing course, and tha ACT Englisa score as second oest. Number of^ 
errors, however, did characterize the remedial writers, Wai^e the ACT 
English and the preliminary halistic rating are worKaDle for entrance 
testing, they are not the best combination for detaruiining freshman 
composition entrance for students who have spent a semester under 
remediation cr who are testing for a second tinie. rherefore tht5 
university is using a holistic essay rating plus an error cuunt on 
that essay as a freshman composition entrance test for studfeats who 
are testing for the second time. (HOD) 
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Defining the Basic Writing Student by Coiint 

Teachers of remedial composition courses knov; unskilled 
writing when they see it. It is characterized by numerous 
errors--in spelling, tense markers, noun olurals, pronoun use, 
sentence boundary markers, utc.^-and is generally shorter, less 
specific, less logical, and more disorganized than the writing 
that is considered to be of college level. Writers who produce 
essays of this quality are regarded as candidates for remedial 
writing 'classes. But when you have 4000, or even 300^ students 
entering school in the fall, how will you determine when the 
writers are so unskilled that they should be placed in your 
remedial course? What measurta or measures will you use? What 
will be the cut-off point indicating need for remediation? 

Various schools have dealt with this problem in different 
ways. In an effort to answer some of Lhese questions at Illinois 
State University, I and my colleagues have been researching 
placement procedures over the last two or three years. Central 
to our investigation has been the relationship of error to writing 
quality , 

The research began with a study that compared students 

in a freshman composition class with those in a remedial writing 

class. The sample was small and the generalizability limited, 

yet this pilot study did lead to further research, , The two 

'$ 
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classes v;are compared on T--unit lenath^ clause lengthy writing 
aoDirehens ion / and errors on a revision passaqe. Oriainal writinq 
was not compared. Of the measures smplDyed, the two clashes 
differed only on error count . In roughly 150 words , the remedial 
class averaged 6.67 errors ^ while the f re.ihman composition class 
averaged 1.28, the difference being significant beyond the .001 
leve 1 , 

This difference is not surprising^ since unskillful 
writing is known to be characterized by error, and remedjation 
correspondingly aims for reduction of errors. The literature in 
the field likewise assumes this characteristic. Slotnik and 
Rogers, for example-/ in applying factor analysis to a large corpus 
of National Assessment papers, found only two significant factors 
differentiating good and poor papers: number of errors and essay 
length. The City College of New York at one time regarded a 
five percent error count as indicative of a need for remediation 
(as reported by Barbara Quint Gray) , and LaGuardia Community 
College eight errors or more in 300 words (this reported by 
Roberta Matthews). Shaughnessy reports that remedial writing 
contains ten to thirty errors in 3 00 words, while Labov observes 
that readers will tolerate only two percent errDrs, 

Acting on our preliminary findings and on published re- 
search, our Freshman Writing Committee devisea a,, set of error 
criteria for further investigation. These criteria divided 
errors into those we considered more serious (sentence boundaries, 
subject^verb agreement, tense markers, noun plural markers) 
and less serious (spelling^ pronoun case and reference, apos- 
trophes, and omitted words) * Each occurrence of the more serious 
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errors was counted, but only one occurrence of those regarded 
as less serious. 

These error criteria were then applied in a multiple re^ 
aression analysis of the writing of two freshman composition 
classes. We counted errors on an in=class essay written at the 
beginning of the semester. The criteria were found to have a 
.4 73 correlatiOi* with final grades- Employing the multiple re- 
gression predictability formula, we found that the error count 
could be no greater than 3,36 in order to assure a reasonable 
chance for a final grade of C, In additional informal checks 
made by teachers of the remedial classes, the students who ended 
the coursa with a passing grade had in general fewer than four 
errors. 

All this preparatory analysis led up to the full-^scale 

research we conducted in the fall cf 1980 in order to establish 

competency--based entrance requirements for freshman composition. 

Our initial procedures were to randomly select a sample of six 

'j 

freshman comp and four basic writing classes. From these ten 
classes we gathered, at the beginning of the semester and at 
the end, an impromptu in-^class essay, a revision of Kellogg Hunt's 
"Aluminum" passage, and a cloze test. (*' Aluminum-' is a series 
/of short sentences to be rewritten in a coherent paragraph. Using 
it has the advantage of demanding the same tasks of all students. 
The cloze test was a short essay with every seventh word omitted, 
to be inserted by the student.) 

The cloze tests were, scored for the approximate word, 
and the "Aluminum" revisions were scored for errors as listed 



on our error critaria. The essays ^ both pris and post, were rated 
at the end of the semester by three trained readers who had no 
knowledge of time of writing^ course represented, or scored or 
the other readers. Then the essays were submitted to an error 
count using our criteria. We also gathnred ACT.-Enalish and Social 
Studies scores and several reading s 3* All this data v^as 
computer-analyzed by the multiple regression procedure, with 
the final holistic rating as our criterion variable. 

Our error counts ditt not come out as the best predictors* 
Injtead, v;e have the preliminary holistic rating as the begt 
indicator of success in our freshman writing course, and the ACT 
English score as second best. Together, their r2 accounts for 
48 percent of the variability in the final essay rating. So we 
are adopting theise two measures for determining entrance into 
our freshman composition course. 

Number of errors, however, did in this study, as in ear- 
liar rasearch, characterize the remedial writers. At the begin- 
ning of the semester their essays averaged 4.06 errors per 150 
words in contrast to 1.89 for the freshman copip students. At 
the end of the semester, the remedial writers had reduced their 
errors to a m.ean of 2.88, a decrease significant at the .001 
level, while the freshman comp students had dropped to 1.48, a 
slightly less significant decrease. As far as. the essay ratings 
are concerned, our remedial classes increased^ again at a more 
significant level than our freshman comp students (beyond .001). 
While those improvements are encouraging in that our remedial 
Gourses are producing the effects we have aimed for, it is, all 
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the 



same, a r 



ant , since the Statii of Illinois 



has recent Iv 



i of all remedial courses at the 
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md the courses are already being 



o ha sad out. 



While: 



ratings and the ACT English scores were 



adopted as oi 



placement measures , we do still have a 



use for the e 



nd error count combination . 



Errors on the 



preliminary essay did have a significant correlation with the 
final essay rating, a coefficient of .51. This score in combina- 
tion with the preliminary holistic rating accounts for 44 percent 
of the variability in the final essay. Not as high as the 48 
percent for essay and ACT scores^ but respectable all the^same^ 
and v/e can use it. While the ACT English and the preliminary 
holistic rating are workable for entrance testing, th^ay are not 
the best combination or determining freshinan composition entrance 
for students who have spent a semester under remediation, or who ' 
for other reasons are testing for a second time. We need a way 
of knov/ing if these students have made enough progress to enter 
the _ regular course, and using the ACT presents problems. Whereas 
the student ^s academic level may have changed over the semester, 
the available ACT score would be the same as at the beginning 
of the semester. 

We therefore have gone to using a holistic essay rating 
plus an error count on that essay as a freshman composition 
entrance test for students who are testing. for the second time. 
What we will do is set up a committee of writing teachers, who 
will decide on a topic for the impromptu essay and, after a 
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training session on reliable holistic rating, will read and score 
the essays, finally countinq errors in the first 150 words. 
This cominittee will determine; whether the students tested have 
re-iched a level of competence for entering freshiTian composition. 

So, while error count was not the best predictor of suc- 
cessful completion of our freshman composition course, its high 
correlation with essay ratings supports its use as an additional 
measure of writing competence. 



■ / 
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SAMPLE GRADING SHEET FOR 101 ENTRANCE EXAM 



STUDENT^ S NAME 

TEST DATE 



GRADER'S NAME 



Sentence boundary probl ems (count sac h error ^ 
A. Fragments ■ • ^ 1 



; B. Comma splices and run-jns 



1 



t\ Faylty word forms (count^ each error) 

A, Plural markers (nouns) 1 

B . Tense markers 

1 . -ed^ for past 

2. irregular verb forms 

3. 3rd person singular present 

1 

3. Subject verb agreement (count each error) 

1 



Total 



(three or fewer errors) 101 



i. I 



189.50 



4 

4 



(If al l errors are Below linea 101 is recommended )' 

4. Pronouns (count errors only once) 

A. Case 1 

B. Number agreement 1 

C. Reference 1 

5. Word mechanics 

A. Omitted words 1 " 

B. Misspelled words ° ' 1 

C. Apostrophe missed or misused 1 
Recommend : 
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TABLE 1. CORRELATION COEFFICIENTS OF SELECTED PREDICTOR 
VARIABLES AND POST 'HOLISTIC RATING OF ESSAYS 



n=103 ACT-E ACT-SS AI.Err. Cloze Es'.Err. Es.Rat.I Es.Rat.II 

ACT English 1.000 .620 -.525 -.738 -.489 .649 .611 

ACT Soc.St. 1..00Q -.1 77 -.51 9 -.275 .348 .254 

"Al." Errors . 1.000 ' .542 .543 ' -.527 -.467 

Cloze Errors " 1.000 .477 -.650 -.527 

Essay Errors . • : 1.000 -.640 -.505 

Essay Rating I . • 1 .000 .549 

Essay Rating II ' , 1 .000 

p<.05 when R>.195. _ ' 

p<.01 when R >'.254. 



TABLE 2. RECOMMENDED SCALE OF ESSAY SCORES AND ACT ENGLISH SCORES 
FOR DETERMINING ENTRANCE INTO ENGLISH J 01 



Essay Score ACT English Score 



(C) - 


3 




2.85 




10 


2.54 




12 


2.43 ' 




14 


2.32 




16 


2.22 




17 


(D)' 


18 




1 .80 




20 


1.58 




22 


1 .37 




. 24 


1.16 " 




25 


(F) 


28 





Applying this table, we would require a student with an ACT English score of 
8 to have an^ essay score of 3. An essay score of 2 must be accompanied by 
an ACT English score of 18, and a„ student sconing has little chance of success 
without an ACT score of 28, - 



